Described below are methods and devices for providing a reconstructed image.
In the past years picture formats requiring to be coded have been steadily increasing in size due, for example, to the introduction of novel types of recording systems. Thus, one of the changes taking place at the present time is a transition from a television transmission system known as PAL (Phase Alternation Line method), which has been used in Europe over the last 50 years and is based on a frame size of 625×576 pixels, to an HDTV (High Definition Television) format with 1920×1080 pixels or 1280×720 pixels. It is to be expected that in the future even larger image formats will be introduced in new types of television systems.
HDTV and future systems use digital compression methods in order to compress a sequence of video images (frames) in such a way that these can be transmitted for example over the internet or via mobile communication channels. However, the increase in size of the frame formats has also led to a considerable increase in the computing power required in order to compress the video data sequence and in the amount of storage space required for this. A consequence of this is that there is also a considerable increase in data transfer between memory and computing units that implement the compression methods.
For this reason study groups such as the Joint Collaborative Team on Video Coding (JCT-VC) for example, a joint working party of the ITU and the ISO/IEC (ITU—International Telecommunication Union, ISO—International Standardization Organization, IEC—International Electrotechnical Commission) are working not only on improving the compression rate, but also on standardized methods to enable video images to be stored efficiently in reference frame buffers of the respective codecs and accessed in a manner that is economical in terms of resources.
FIG. 1 shows a known device for compressing a sequence of frames, the device having a reference frame buffer SRB. In this case frames are coded for example by a predictive coding scheme, also known as inter coding mode. One of the frames is decomposed into frame blocks BB, for example 16×16 pixels, and is subsequently encoded frame block by frame block. For one of the frame blocks a search is then made for a reference frame block RBB in a reference frame REF which provides a good basis for estimating a frame content of the frame block. For this purpose the frame block is passed to a motion estimation unit ME which, on the basis of a reference subframe REFT that includes parts of the reference frame REF following a frame decompression by a frame decompression unit PC, selects the reference frame block from the reference subframe and signals the selected reference frame block to a motion compensation unit MC by a motion vector MV. The motion compensation unit provides the reference frame block on the basis of the reference frame and the motion vector.
Next, a difference frame block BD is generated by subtracting the reference frame block RBB from the frame block BB. The difference frame block is subsequently subjected to a transformation in a transformation unit T, for example in accordance with a discrete cosine transform method. At the output of the transformation unit there are transform coefficients TK available which are subsequently supplied to a quantization unit Q for quantization. At the output of the quantization unit there are quantized transform coefficients TQ available which are converted into an output signal AS by entropy coding performed by an entropy coding unit EC.
The quantized transform coefficients TQ are converted in a feedback loop into reconstructed transform coefficients TKR by an inverse quantization performed by an inverse quantization unit 10. The reconstructed transform coefficients TKR are transformed into a reconstructed difference frame block BDR by inverse transformation by an inverse transformation unit IT. Following this a reconstructed frame block RBM is generated by adding the reconstructed difference frame block BDR and the reference frame block RBB.
In older coding methods the reconstructed frame block is written directly into the reference frame buffer. In methods currently undergoing standardization, in order to reduce a data volume the reconstructed frame block is initially subjected also to frame compression by a frame compression unit PC which significantly reduces the data volume of the reconstructed frame block. A compressed reconstructed frame block RBC produced as a result of the frame compression unit PC is subsequently stored in the reference frame buffer. In order to allow the motion estimation unit and the motion compensation unit access to the required frame data, when a reference frame REF or a specific detail of the reference frame is requested the respective compressed reconstructed frame block is first read out from the reference frame buffer SRB and converted into a reference subframe REFT by frame decompression performed by a frame decompression unit PD.
FIG. 2 shows a decoder corresponding to the encoder shown in FIG. 1. In this case the output signal AS is decoded into quantized transform coefficients TQ by an entropy decoding unit ED. Furthermore the quantized transform coefficients are inversely quantized into reconstructed transform coefficients TKR by the inverse transformation unit IQ. This is followed by an inverse transformation of the reconstructed transform coefficients TKR into a reconstructed difference frame block BDR by the inverse information unit IT.
In addition to the output signal the respective motion vector MV, inter alia, is also transmitted to the decoder. From this, using the reference subframe REFT, the decoder can determine by the motion compensation unit MC the reference frame block RBB, which is converted into the reconstructed frame block RBM by addition with the reconstructed difference frame block.
The reconstructed frame block RBM can be visualized for example on a display. The reconstructed frame block RBM is subsequently converted by a compression performed by the frame compression unit PC into the compressed reconstructed frame block RBC, which is then stored in the reference frame buffer SRB. The compressed reconstructed frame blocks stored in the reference frame buffer can be decompressed into the reference subframe by the frame decompression unit PD.
Chong Soon Lim's article “Reference frame compression using image coder”, ISO/IEC JCTVC-B103 2nd Meeting, Geneva, July, 2010, describes a lossless frame compression method/frame decompression method in which bit-plane coding is performed following a floating-point DCT transform (DCT—Discrete Cosine Transform) and scanning of coefficients in a one-dimensional representation, arranged two-dimensionally after the transformation.
In an article by Mehmet Umut Demircin et al., “Compressed Reference Frame Buffers (CRFB)” ISO/IEC JCTVC-B089 2nd Meeting, Geneva, July 2010, a buffer memory access bandwidth reduction technique was proposed. In this case, as well as a transformation and quantization, a DC prediction and entropy coding are proposed for the frame compression unit PC or an inverse thereof for the frame decompression unit PD.
In Madhukar Budagavi's article “ALF memory compression and IBDI/ALF coding efficiency test results in TMuC-0.1”, JSO/IEC JCTVC-B090, test results for compression and decompression of frame data upstream and downstream, respectively, of a deblocking frame memory are presented.
Hirofumi Aoki's article “DPCM-based memory compression”, ISO/IEC JCTVC-B057 2nd Meeting, Geneva, July 2010, finally, a one-dimensional DPCM-based frame memory compression method (DPCM—Differential Pulse Code Modulation) is presented.
At least the compression methods proposed in Lim's and Aoki's articles are lossless.